Feedback minimized optimum filters and predictors

ABSTRACT

A nonlinear adaptive signal processor is provided wherein during a training phase, based upon a prior knowledge of the desired processor response to a given input, the output signal preceding the contemporary value of the output signal is employed in the feedback sense for minimizing storage required of the desired responses of the processor which, when trained, responds as desired to a different input having the same statistics as the input employed in the training phase.

United States Patent 'l'rxlw lnulrumenls Incorporated Dallas, Tex.

['I l] Amzigncc [54] FEEDBACK MINIMIZED OPTIMUM FILTERS AND IMO/172.5340/l72.5

3,3ll,895 3/]967 Clapper 3,3l7.900 5/1967 Clapper PREDICTORS l 1 Claims,12 Drawlng Figs. Y Us L ABSTRACT: A nonlinear adaptive signal processoris pro- [52] .C 340/1725 vided wherein during a training phase basedupon a prior [5 u Cl 15/18 knowledge of the desired processor responseto a given input, [50] D'SQQIC'I the output signal preceding thecontemporary value of the 57 output signal is employed in the feedbacksense for minimizing storage required of the desired responses of theprocessor [56] References Cited which, when trained, responds as desiredto a different input UNITED STATES PATENTS having the same statistics asthe input employed in the training 3,267,439 8/l966Bonner......................... 340/1725 phase.

i l 22 MEMORY 23 T /4 UPDATE 23 v QUANTIZER L DIVIDER 32' STORAGE I? "iI x E QUANTIZER u' u' I80 v STORAGE V QUANTIZER SOURCE us "i I I! I0 27UNIT SOURCE PATENTED AUG! 0 men SHEET 1 OF 5 J 2 O I 2 5 Y E 4 N m D R EF l E MU m u\|/ P WI W l\ M 3 D 8 2 a 0 ma 2% 2 2 2 E m E L Z. A U P uwH E E 7 Y 6 RE A 7 N 2 0A R R mm m m MU S S B R R R E n EA m U H w T N NNi A l M x U W U Q 5 mm H T1 7 W 1 E Cm R U 0 S E T. RM N W M A L m & 4000 UR 8 M O OS S T E L m w L 0 T4 IOOOV U W a: U.l 0 3 *V 00] E N H UVw o) o a V a NY m! Hi" PW, IIIOI W ATTORNEY FIG. 4

PATENTED AUG u 0 [9H SHEET 2 BF 5 W11: 1,1: 4 8 AM 5 E 11 M lt A u E 5$1: i? wit: M 2: 0 v0 9. a 4 9 PATENTEU AUEIOIQTI 3,599.15?

SHEET 5 BF 5 DESIRED ACTUAL. OUTPUT INPUT ouTPuT F P OF PLANT CONTROLLERTO PLANT PLANT 0 LAN z(t) u(t) x(t) \BOO \3OI OPTIMAL PLANT NONLINEARPROCESSOR 302 :0) VIII WI) 1 NOISE COMBINATION I fl) E MQ x(t) I PROCESSPROCESS l PROCESSOR 3o4 3o5 SIGNAL 0) I 307 2(t) s(t) L PROCESS \303 I306 'n(t) T NOISE COMBINATION um gzlh'j''h l PROCESS PROCESS I PROCESSOR\303 305 DELAY I I s IGNAL U) T 2(t) sh) L2??? 2: 308

DELAY I l OPTIMAL NOISE COMBINATION u(t-A) XII) I T IIIIAEII psRlgNAs h)307 z(t) s(t) CE 3o6 modeling, and classification.

I-Ieretofore. linear processors have been widely used. The design ofoptimum linear processors and predictors according to a mean squareerror criterion is.well known in the art. Such systems theoreticallydiscussed by Norbert Weiner in his work entitled The Fourier Integraland Certain of its Applications published in 1933 by Dover PublicationsInc. Weiner has also discussed the realization=of a nonlinear system.The theoreticalaspects of the Weiner theory of nonlinear systemcharacterization may be employed in the manner disclosed in the thesisof Amar G. Bose, Massachusetts Institute of Technology, June, 1956,entitled A Theory of Nonlinear Systems," reproduced in. M.l.T. ResearchLaboratories of Electronics'Technical Report No. 309.

The use of the Weiner theory is further disclosed by Bose in Systems andmethods described and claimed in U.S. Pat. No. 3,265,870 Aug. 9,1966 toBose.

The present invention makes more practical and realizable the Weinernonlinear processorv asimplemented by Bose. Applicant provides forminimization of the instrumentation necessary to apply the Weiner-Bo'seteachings to challenging signal translation problems of the'present dayin the fields specificslly'mentioned above(identification....elassification).

The U.S.Pat. No. 3,265,870 toBose makes clear that, as the complexity ofthe processor is'increased, thenumber of filter coefficients necessaryto specify the processor grows very. rapidly. In the Bose patent use wasmade of Laguere ne' tworks'when s Laguere coefficients are employed andn gate functions are-usedfor each coefficient, there are n' coefficientsto evaluate. In the Bose system such coefi'rcients were represented bycharges stored on capacitors. Thus the number of coefficients to'be sorepresented is further to be increased when the number of gate functionsemployed in Bose is increased to quantize into smaller cells thefunction space of the input signal. Bose pointed out that the latterrefinement is desirable from the point of view of reducing thefilteringerror but that it leads to a very large number of coefi'rcientsto evaluate. Bose then indicates steps that might be taken to reduce thenumber of such coefficients in the case where a priori information isavailable relative to 'the character'of the input signal.

The present invention relates to minimizing system complesity beyond anylevelpostulated by Bose but without sacrificing the capability of thefilter.

In accordance with the present invention an arrangement is employedwherein the processor output signal produced in an output utilizationcircuit isapplied to a unique feedback structure whereby the number ofcoefficients to be evaluated is reducedlfl'hus, the improvementcharacterizing the present invention is appreciable in anumerical senseas well as in actual reduction of the amount of hardware necessary toemploy an optimal nonlinear processor. This is accomplished by use of afeedback network wherein at least one signal, the signal x whichprecedes the contemporary value x; of the processor output, isemployed..The use of feedback further reduces the number of trainingpoints required in the training phase of operation.

In accordance with the invention successive time samples of an inputfunctionu are quantized'toproduce members of successivegroups of addressfunctions. for controlling a gating operation which in response to eachof the groups will produce an independent gating output signal. Thegating output signal related to the contemporary value u, of the signalu is combined witha second signal z representative of the contemporaryvalue of the desired processor response x, to the input u and with athird signal x,,.. The resultant combined signal is stored at theselectedaa'ddressand, further, is normalized and combined with said.third signal -.r to form the processor output signal x,. At the sametime the third signal -x,,, is quantized to produce a second member ofeach of the groups of the functions for controlling the gatingoperation. In

general, the input ui, desired output z and previous actual" output xeach represent a sample of a single valued time function or amultiplicity of such functions, in which case they may be represented asmulticomponent vector quantities.

In a system sense, the processor includes at least a pair of levelselectors one of which senses the signal :4 to provide an outputrepresentative of the contemporary value u, of the signal u. A pair ofstorage arrays are each addressed in one coordinateby one of the levelselectors and each is addressed in a second coordinate by the other ofthe level selectors. Means are provided for summing'a desired outputsignal z, with the contents of one of said storage arrays at theselected address and with the negative of a third signal at for storingat the selected address acondition representative of the sum. Means areprovided'for'normalizing the sum and for combining withthe normalizedsignal the third signal. Means for generating the third signal includesa delay means responsive to the processor output with circuit means'forapplying the delayed signal to the second of the'level selectors.

For a more complete understanding of the present invention and forfurther objects and advantages thereof, reference may now be'had to thefollowing description taken in conjunction with the accompanyingdrawings in which:

FIG. 1 is a block diagram of an embodiment of the present invention;

FIG. 2 is a diagram illustrating ananalog form of an embodiment of theinvention;

FIG. 3 illustrates a four-level quantizer circuit;

FIG. 4 is a table illustrating the operation of the circuit in FIG. 3;

FIG. 5 illustrates a rnulti-input multifeedback processor in a plantidentification environment;

FIG. 6 illustrates a more general multi-input multi-output systemembodying the invention;

FIG. 7 illustrates" a processor which trains and operates in response todigital signals; I

FIG. 8 diagrammatically illustrates the production of a plantcontroller;

FIG. 9 diagrammatically illustrates application of the plant controller;

FIG. 10 diagrammatically illustrates a filtering process;

FIG. 11 diagrammatically illustrates a smoothing process; and

FIG. 12 diagrammatically illustrates a prediction process.

The system shown in FIG. 1 provides for nonlinear processor' which maybe trained for optimum processing of single valued time varying functionucharacterized by two components 'u(t) and [u(I)-u(t-T) in the mannerdisclosed in the generic sense in the U.S. Pat. No. 3,265,870 to Bosebut with provision for reducing the required storage by many orders ofmagnitude. The use of a bar under a given symbol e.g. u herein signifiesthat the signal so designated is a multicomponent signal, e.g. u(t) andu(t) u( t-T). The reduction is accomplished through the use of afeedback operation which at any one instant requires only one sample ofeach of the two components of I the input signal u and. thus materiallyminimizes the storage problem. As in the Bose patent, the processor istrained in dependence upon some known or assumed function which is adesired output such that the output function 'x' corresponds with z forinputs having statistics similar to u. Thereafter the processor willrespond to signals u, u", etc. in an optimum manner. In the followingdescription, the training phase will first be discussed following whichthe changes to carry out operations on signals other than that used fortraining will be described.

TRAINING In FIG. 1 the first component of signal u from a source formsthe input to a quantizer 11. The output of quantizer 11 is connected toeach of a pair of storage units 12 and 13. The storage units 12 and 13will in general have like capabilities and will both bejointly'addressed by signals in the output circuits of the quantizer 1 1and quantizers 14 and 15. The

storage units 12 and 13 are multielementstorage units capable of storingdifferent electrical quantities at a plurality of different addressablestorage locations.

Briefly, the third quantizer 15 has been illustrated also addressingboth storage units 12 and 13in accordance with the second component ofthe signal it derived from source 10, the delay 18 and the inversionunit 18a. More particularly, if the signal sample u, is the contemporaryvalue of the signal from source 10 then the input applied to quantizer15 is ur-u This input is produced by applying to a summing unit 17 u,and the negative of the same signal delayed by one sample increment inthe delay unit 18. For such an input, the storage units 12 and 13 may beregarded as three dimensional matrixes of storage elements. In thedescription of FIG. 1 which immediately follows, the quantizer 15 willbe ignored and will be referred to later.

The output of storage unit 12 is connected to an adder 20 along with theoutput of a unit 21 which is a signal 2 the contemporary value of thedesired output signal. A third input is connected to the adder 20 from afeedback channel 22, the latter being connected through an invertingunit 23 which changes the sign of the signal.

The output of adder 20 is connected to a divider 24 to apply a dividendsignal thereto.

The divisor is derived from storage unit 13 whose output is connected toan adder 26. A unit amplitude source 27 is also connected at its outputto adder 26. The output of adder 26 is connected to the divider 24 toapply the divisor signal thereto. A signal representative of thequotient is then connnected to an adder 30, the output of which iscontemporary value x, the processor output. The adder 30 also has asecond input derived from the feedback channel 22. The feedback channel22 transmits the processor output signal x, delayed by one unit timeinterval in the delay unit 32, i.e., 1 The feedback channel 22 also isconnected to the input of the quantizer 14 to supply the input signalthereto.

A feedback channel 36 leading from the output of adder 20 to the storageunit 12 is provided to update the storage unit 12. Similarly a channel38 leading from the output of adder 26 is connected to storage unit 13and employed to update memory 13.

Briefly and during the training phase and neglecting the presence ofquantizer 15, the system operates as will now be described. Thecontemporary value u; of the signal it from source 10 is quantized inunit 11 simultaneously with quantization of the preceding output signalx (which may initially be zero) by quantizer 14. The latter signal isprovided at the output of delay unit 32 whose input-output functions maybe related as follows:

7' is the delay in seconds,

xr-xUT-HJ, and

m' IU-U 'RL where i is an integer, T is the sampling interval, and r, isthe time of the initial sample. The two signals thus produced byquantizers ll and 14 are applied to both storage units 12 and 13 toselect in each unit a given storage cell. Stored in the selected cell inunit 12 is a signal representative of previous values of the output ofadder 20 as applied to this cell by channel 36. Stored in thecorresponding cell in unit 13'is a condition representative of thenumber of times that that cell has previously been addressed, thecontents being supplied by way of channel 38. Initially all signalsstored in both units 12 and 13 may be zero. The selected stored signalsderived from storage array 12 are applied synchronously to adder 20along with z and x signals.

The contemporary output of adder 20 is divided by the output of adder 26and the quotient is summed with x in adder 30 to produce thecontemporary processor response x The contemporary value x is dependenton the contemporary value 14, of u, the contemporary value 2 of thedesired output z and negative of x i.e.: (x as well as the signals fromthe addressed storage cells.

It will be found that upon analysis the system functions in a mannerwhich may be described in the following terms involving a differentialequation in vector form. A nonlinear system may be characterized by avector differential equation HUM That is, it is the time derivative ofthe signal x and is a vector valued function g of x, u, and t. Thefunction g in a theoretical sense describes the system and may allow thefirst order vector valued differential equation (1) to represent asystem governed by a differential equation of multiorder. In importantcases of stationary systems, time does not appear as a parameter of g.This will be the case for the systems considered first herein.

By choosing the time increment Tsufficient small, the variation of x(t)can be made small in any interval (1, r-T), subject only to reasonablephysical constraints. Then Equation (1) can be approximated F go hu D' mHere the problem is to determine the first element on the right ofEquation (2). The feedback provision of the present invention can beshown to proceed for detennination of the optimum operation T3 ofEquation (2).

The functions above described in connection with the system of FIG. 1provide the signal x, automatically.

POST TRAINING OPERATION The system shown in FIG. 1 establishes voltageconditions which represent the optimum nonlinear processor for treatingsignals having the same statistics as the signal z(t) upon which thetraining is based.

After the system has been trained based upon the desired output 2 over astatistically significant sequence of u and z, the switches 21a, 23a and27a may then be opened and a new input signal 14 employed whereupon theprocessor operates optimally on the signal u in the same manner as abovedescribed but with the three signals z x and unity no longer neededwithin the update channels.

In the system as shown in FIG. 1 quantizer 15 provides an outputdependent upon the differences between sequential samples in and uemploying a'delay unit 18 and a polarity reversal unit 18a. In thissystem a single delay unit 18 is provided at the input and a singledelay unit 32 is provided at the output. In general, more delays couldbe employed on both input and output. However, physical considerationswill generally require that there will not be required more delay unitson the input than on the output. Further, if the system is responsive topass low frequencies selectively, relative to high frequencies; thenfewer delays would be required on the input than on the output. In useof the system with quantizer l5, storage units 12 and 13 mayconveniently be regarded as three dimensional. Of course, elements ofinput vector and output vector need not be related by simple time delaysas will be subsequently shown in FIG. 6.

FIGURE 2 A more detailed embodiment of the invention is shown in FIG. 2.The third quantizer 15 of FIG. 1 has been eliminated. Storage elements12 and 13 are depicted as two-dimensional arrays of storage elements inthe form of electrical capacitors. Where appropriate, parts of FIG. 2have been given the same reference characters as in FIG. 1.

In FIG. 2 electrical charges are placed on the capacitors which are, inpart, selected by voltage states on lines 41-44 leading from quantizer11 Lines 41-44 extend vertically through the arrays of storage elementsin units 12 and 13. The capacitor selections are then completelyspecified .by voltage states on lines 4548 leading from quantizer 14.The latter lines extend horizontally through the storage units 12 and13. Quantizers 1'1 and 14 operate asdisclosed in US. Pat. No.

3,265,870.to Bose so that only one of the lines from each quantizer willbe energized at any one time. Quantizers 11 and l4 'are controlled by atimer 37. Each storage element is provided with controlcircuits tocontrol the signal to be transmitted from a given capacitor and thecharge placed on a given capacitor.

The storage units 12 and 13 areillustrated as comprising a 4X4 arraywhich may form a part of larger arrays asindicated by the dottedterminations of the electrical grid conductors.

' In storage unit 12, capacitors 51-66 are employed. In storage unit 13,capacitors 7l86+are employed. One pair of capacitors is selected oraddressed each-time a sample of the input signal u, is employed,selection being by the outputs of units 11' and 14. They are addressedby enabling AND gates such as gates 90 and 91. Gates 90 and 91 each haveone input connected to bus 41 leading from=quantizer 11 and anotherinput connected to bus 45 leading from quantizer 14. The outputof gate90-controls a switch'a'ctuator 92 which serves, to sequentially causeclosure of switch terminal 93 and then switch'terminal'94. Similarly,gate :91 controls a switch actuator 95 which operates to close's witchterminal 96 and then switch terminal 97. Closure of switch terminal 93applies the voltage on capacitor 51 to adder 20 each time the AND gate90 islenabled. Following the latter action, switch terminal 94 is closedto apply to the capacitor an'additional charge proportional tothevalueof the voltage at the input to divider 24 to update the quantity storedon capacitor 51.

Gate91 is enabled coincident with AND gate 90, thus causing switchterminal 96 to close. Thisapplies voltage to adder 26 proportional tothe charge,on: ca' pacitor 71. Thereafter,

'switch terminal 96 is opened and-switch terminal 97 is closed to updatethe charge on capacitor 71 by applying thereto a voltage "representativeof the output of adder 26. The output ofadders 20 and 26 are appliedtodivider 24. The output of divider 24 together with the output of delayunit 32 is then applied to adder 3 0 whose output is the desired filteroutput .t,-. It

will be noted that the timer 37 controls the sampling'inte'rval inquantizers 11 and 14-and also controls the delay unit 32. 1

In operation, a voltage is placed on capacitor 51 each time itis-addressed during training. Capacitor 51 in the storage unit 12 willhave increments of charge supplied thereto each time it is addressedwith such increment being dependent upon the quantity x and the valuez,'.

In contrast, the capacitors in storage unit 13 will be incremented, eachtime addressed, by a voltage representative of unity, i.e., the signalfrom source 27. Thus, the summation signal from adder 20 is normalizedby dividing the signal by a voltageproportional to the number of timesthe given storage location has been addressed. I

The switching unit represented by unit 92 having terminals 93 and 94 maybe a conventional stepping switch which will complete its cycle inresponse to an input as from gate 90 in a time interval lessthan theperiod of the timer 37. Such units are well known and are commonly'used.

The system shown in FIG. 2involves only the two inputs :1,

and x,,,. for memory addressing purposesand utilizes only a single timedelay 32 at the output. Where desired additional inputs can be employed.such as the signal u,u, shown in FIG. 1. v An additional feedback withan additional time delay such as the delay 32' may also be employed toprovide a feedback of function x Additional delays may provide x x etc.Thus, whilethe system of FIG. 2.has been described in detail, it is notintended that the invention, be limited to a system where only twoquantizers 1 1 and 14 are employed. An addiand another quantizer will beemployed for each additional feedback function employed.

It will be'seenthat, as to the input signal, the feedback employed inFIGS. 1 and 2 requires only consideration of the contemporary value ofthe elements of u and thus permits formulation of a nonlinear processorwith minimal storage. It is from this fact that significant advantageover the Bose system is achieved.

FIGURE 3 The quantizers 11 and 14 may be of the type shown in FIG. 3.For example, if the source 10 is connected by way of an AND gate 10a ascontrolled by timer 37 then a voltage will be produced on only one ofthe lines 4144 depending on the amplitude of the signal at the instantthe timer opens gate 100. The quantizer'comprises three inputtransistors 101, 102 and 103. The bases of transistors 101-103 aresignal energized through Zener diode units l04-106, respectively, all ofwhich are connected to the output of gate 10a. The transistor 101 isconnected at its emitter through resistor 107 to ground and to line 44.The emitter is also connected by way of resistor 108 to the base oftransistor 109 which is connected in-parallel with the transistor 110.The emitters of transistors 109 and 110 are connected to ground. Thecollectors are connected to line 43. The base of transistor 110 isconnected to the collector of transistor 102 whose emitter is grounded,and to the base of a transistor 112. Line 43 is connected to the supplysource +Vcc by way of resistor 113 and by way of resistor 114 to thecollector of transistor 112. The juncture between resistors 113 and 114is connected to the collector of transistor 101 and, by way ofresistor115, to line 42 which is common to the collectors of transistors 116 and117.' The base of transistor 116 is connected to the collector oftransistor 112. The base of transistor 117 is connected by way ofresistor [18 to line 41 and to the collector of transistor 103. Theemitters of transistors 1'16 and 117 are connected to ground.

In operation, if the signal from unit 10a is less than the breakdownvoltage of unit 106, transistor 103 is off, its collector is at thesupply potential and thus line 41 is at high potential providing a oneoutput. Since the collector of transistor 103 is at high potential,transistor 117 conducts, providing a voltage drop across resistor 115 sothat line 42 is essentially at ground potential or zero."

In a similar manner, transistor 101 and 102 are off. This means that thebase of transistor 110 is at high potential thus conducting so that line43 is substantially at ground potential. Since transistor 10] is'notconducting, line 44 islikewise at ground potential.

When the input signal exceeds the breakdown potential of unit 106'butdoes not exceed the breakdown potential of units 104 and 105, transistor103 conducts so that line 41 is at ground potential. Driving the base oftransistor 117 to ground stops conduction therein so that line 42 ishigh. The circuit involving transistors 116 and 117 is a NOR circuit.Since transistor 102 is not c'onductingthe transistor 112 is conductingso that its collector is substantially at ground causing transistor 116to be nonconductive. Likewise, transistor 102 causes transistor 110 toconduct placing line 43 at ground potential. As before, transistor 101is not conducting and line 44 remains at ground. Thus only line 42 ishigh.

The sumecircuit operation can now readily be visualized for the casewhere the signal exceeds the breakdown potential of unit 105 but not l04and for the case where the input signal exceeds the breakdown potentialof unit 104.

It will thus be seen that the thresholds, or breakdown potentials forunits 104-106 are selected in dependence upon the desired quantizinglevels.

FIGURE 4 a As indicated in'the table of FIG. 4, if theinput voltage inFIG. 3 is less than the level V, of diode unit 106, then line 41 will beenergized and the remainder of lines 42-44 will be deenergized; If thevoltage is greater than V, and less than the level V, of diode unit 105,then line 42 only will be energized. If thevoltage is greater than V,but less than the level V;, of diode unit 104, then line 43 only will beenergized. If the voltage is greater than V 'then line 44 only will beenergized. Diode units 104-106 are illustrated as Zener diodes Theydiffer one from another by their threshold voltages which satisfy theinequality V, V, V,. They may comprise single diodes of differentbreakdown voltages or as, illustrated in FIG. 3, a multiplicity of likeunits in series.

FIGURE -IDENTIFICATION source 2]. I

- The processor of FIG. 5 corresponds generally with the processor ofFIG. I in that the unit A includes'units 20, 24, 26 and 30. connected asin FIG. 1, shown for convenience as a single block in FIG. 5.

One application of the multi-input system is indicated in FIG. 5 whereinthe desired output is the output from a plant 2! and the inputs to thequantizers correspond with and are derived from the four inputs to theplant 21. In this system, the processor is trained in real time independence upon the inputs to plant 21 and the output therefrom tosimulate an out put .1 which corresponds with the actual output z of theplant. Once trained, the switches 21a, 23a and 27a will be opened. Inthis condition, the four processor input signals may be varied at willto simulate an outputx which would be representative of the output fromthe plant 21 for corresponding changes of the four input signals. Insuch case the response of the'simulated plant to any particular changecan be observed without the expenditure of time or energy necessary toeffect such changes or to interfere with actual plant operation.Moreover,

these observations can be effected on an accelerated time scale.

nouns s-ctAssrrlcArroN FIG. 6 illustrates an embodiment of the presentinvention wherein a'pl'urality of input signals are employed as well asa plurality of feedback signals and is further characterized byproviding multiple outputs of a more general class than is possibleinFIG. 5. In FIG. 5 multiple outputs can be obtained such as the outputsx,, x x etc. it will be noted that the only signals. applied to addersin FIG. 5 (the adders being shown in FIG. 1) are derived from units 21,23 and 27. In FIG. 6, the two outputs x; and x, are contemporary valuesof the expectation of two desired outputs and may be completelyindependent. FIG. ii is also more general in that it-includes means forvarying the strength of such signals comparable to the outputs of units2], 23 and 27 in FIG. 5 to provide for selective emphasis of certainportions of the training operation over other portions of the trainingoperation.

More particularly, in FIG. 6 input signals u,, in, u and n, are appliedto quantizers 121 124 each of which addresses storage arrays l2a. 12band storage arrays 13a and [3b. Quanrizcrs l 2.5. I26 similarly addresseach of storage arrays 12a and [2b and 13a and lllb in response to twofeedback signals 'ni m- The signals retrieved from storage in responseto addresses 7 specified by the quantizers 121-426 are connected toadders as in FIG. I wherein feedback is employed. More particularly, thesignal from storage array 12a is connected to adder a and the signalfrom storage array 12b is connected to adder 20b. The signal fromstorage array 130 is connected to adder 26a and the signal from storagearray 13b, to adder 26b. The outputs of adders 20a and 26a are connectedto divider 24a. The outputs of adders 20b and 26b are connected todivider 24b. The outputs of dividers 24a and 24b are connected to adders30a and 30b respectively to provide the output signals x, and :q,respectively. The output signal x, is applied to a delay unit 320 whilethe output at; is applied to delay unit 32b. The signal x is thusapplied by way of conductor 22a to quantizer 12.5, to the second inputof adder 30a and to a polarity in verter unit 234. Similarly the outputsignal x is connected to the second input of adder 30b and by way ofconductor 22b to quantizer 126 and to a polarity inverting unit 231;.

The output of polarity inverter unit 23a and the desired signal z; areconnected to an adder 127 whose output is connected to the input of avariable gain amplifier 128 having a gain function g The output ofamplifier 128 is connected to the second input of adder 200. Similarly,the output of complementing unit 23b and the desired signal z, areconnected to the inputs of adder 129 whose output is connected to theinput of a variable gain amplifier 130. The gain of amplifier 130 isindicated as corresponding with the function 'g,. The output ofamplifier 130 is connected to the second input of adder 20b. Theincrementing signals applied to adders 26a and 26b similarly aremaintained under the control of amplifiers 131 and 132, the gains ofwhich are '3, and g respectively.

The system of FIG. 6, in general, will operate in the manner abovedescribed in connection with FIGS. 1 and 2. The multiinput andmulti-output operation may find application in a number of differentphysical problems. One such problem has been indicated in FIG. 6 ascomprising the identification of and distinction between two differentclasses of sea crafts as they travel through the region occupied by agiven listening station. More particularly, the system may comprise anarray of marine acoustic sensors located along a waterway. The foursignals from the sensors thus comprise the inputs to the quantizers12ll24. During training the desired output signal z, would be unitywhile periodically sampling the outputs from the array [35 as tugs areobserved to travel along a plurality oi paths I36, I37 and 138. Duringthis training interval the desired output signal z, will be zero.Further. training would be carried out wherein the second desired outputsignal z, would be unity while sampling the output signals from thearray 135 while detecting signals from a different class of crafts suchas boats driven by an outboard motor are observed ,in the area of thearray 135. Training would be carried out during time intervals when thecraft travel at different headings and different speeds and withdifferent species of the class of motors involved. During this traininginterval the desired output signal z, is zero.

The signals from amplifiers 128, 130, 131 and 132 would be used only intraining. Thereafter, the system would operate in response to varioussignals detected by the array 135. Whenever a craft of the first classpasses the array 135 the output at, would be of unity value, forexample, and otherwise would be zero. Whenever a craft of the secondclass is in the region of the array 135, the second output A; would beunity but otherwise zero. if craft from both classes were present thenboth outputs would be unity. Thus the outputs x, and x, would providefor classification of the character of the vessels in the region-of thearray, 135.

The foregoing example has been given by way of illustration and isintended to indicate the versatility of the processor and its operationand is not in any way limited to the particular on vironmcnt; to thenumber of inputs employed. nor to the number of feedback elementsincorporated in its operation.

It will be appreciated that if g, is equal to g, then the storage array13b could be eliminated along with the amplifier 132 and the adder 26b.In such case the output of adder 26a would be connected to both dividers24a and 24b.

In use of variable gain, an example is shown where the system employsthe signals from the sensors in the array 135 to supply to a gaincontrol unit 141 by way of paths 140. The gain control unit 141preferably operates in the same manner of a during training. When thesignal amplitude is high, the gains 3, and Pg, will be larger than whenthe signal amplitude is low. In such case, greateremphasiswill be placedupon the error, i.e.: the difference between x, and zz for example, whena given craft is close to the array 135 than when it is remote. That is,the training will emphasize the error when the signalto-noise level ishigh. Other criteria of course could be employed in generating andapplying the gain control functions such means in general being wellknown, and dependent upon characteristics other than amplitude.Therefore, the example above is not limiting. v I

FIGURE 7FlLTERlNG FIG. 7 is an embodiment of ..the system in digitalform generally corresponding to that of F IG. 2. In this system theinput u is illustrated as having been stored on a magnetic tape 200 withthe successive words on the tape representing, in digital code, theamplitude of a given input function such as for example a seismogram.The function z is similarly stored on a tape 201 with the successivewords representing the desired output ofa filter to which the inputsignal ris to be applied. The output of the playback head is applied toa decoder 202 with the outputs therefrom being applied to a randomaccess memory. 203. Each memory word may provide for storage of aplurality of bits as in the word segment 204 representing the number oftimes (N) that a given address has been accessed. The word segment 205represents the quantity Q stored in the memory in the course of thetraining of the filter and, in general, will have its counterpart in thesystem of FIG. 2, in the charge on the capacitor such as capacitor 51.The word segment 204 will represent the number of times a given addresshas been accessed and will find its counterpart in the charge ofcapacitor 71, HO. 2.

In operation, the successivewords on tape 201 are loaded into an inputregister 210. In synchronism successive words on tape 200 are loadedinto decoder 202. Decoder 202 is connected to the random access memory203 by means of lines which, in general, correspond with lines 41-44 ofFIG. 2. By means of this connection words in memory 203 are identifiedand caused to transfer to selector register 206.

As a part of the memory, a selector register 206 is provided with anoutput selector 207-and an input selector 208 associated therewith.

The output selector 207 is actuated by a storage register 211 in which adigital word representing the magnitude of x is stored. The effect is tocause to be selected from the group of words storedin theselectorregister 206 that word uniquely designated by the address. componentfrom decoder 202 and the address component from function r The selectedword, comprised of segments corresponding to functions and N istransferred to a register 212 by way of path 213. The word segment inregister 212 representing a particular function 0 is then applied by wayof a gate 214 to an adder 215. The contents of register 210 are alsoapplied to adder 215 by way of a gate 216. Adder 215 is connected toadder 217 to supply one input to adder 217. The second input to adder217 is applied by way of a gate 218 which is energized at one input bythe negative of .m 'appearing on path 219. Adder 217 is connected at itsoutput as the dividend input to divider 220 and, by way of path 221,-.toa section in a selector input register 222. The divisor input-Lto thedivider 220 is provided at the output of adder. 224 one input of whichis encrgized by the gate 223 and the other. byzgate 227. One input ofgate 223 is energized by the portion of the register "212 representingthe stored function N. One input of gate 227 is supplied by the unityincrementing source 225. The unity source 225 is thereby connected tothe second input of adder 224 through gate 227 so that the output ofadder 224 as applied to the register 222 by way of path 226 representsthe signal N+1. v y

The output of divider 220 is connected to one input of adder 230. Theother input of adder 230 is supplied with a signal representing thefunction xm by way of path 23l and gate 232. The output of adder 230appears on path 233 and represents the system output x The system outputx, is supplied by way of a gate 234 and an OR gate 235 to a register236'. The paths 219 and 231 are con nected at the outputs of register236 to provide .r and the complement ofx respectively. I

The selector register 211 is fed from register 236 by way of path 231 toa gate 240. A second selector register 241 is connected at its input tothe gate 240 and, in turn, is connected to the selector 208.

The system hasbeen illustrated with a motor 250 driving the tapes 200and 201 with the tapes being driven in synchronism. The outputs of theplayback heads are applied by way of a gate 251 to a timer 252 therebyproviding a sync pulse each time words stored on the tapes 200 and 201are reproduced. Timer 252 may be of various forms but as shown has sevenoutputs to provide a series of pulses on lines 25325 9 seriatim. Moreparticularly, following each sync pulse and over an interval less thanthe period between sync pulses each of lines 253 259 is pulsed. A firstgate pulse is thus provided on line 253 from the timer output to thedecoder 202 and to a gate 260 whose output is applied to gate 216. Thefirst pulse loads the contemporary value z. of the desired outputfunction 2 into the adder 215 and transfers to the selector register 206those words corresponding with that portion of the desired addressrepresented by the output of decoder 202.

A second gate pulse on line 254 is applied to gate 240 to clock thecontents of register 236 into the selector register 211. The second gatepulse also enables gate 262 thereby loading the same function x backinto the register 236 through gate 235.

The third gate pulse appears on line 255 to apply the contentscorresponding to the function Q from register 212 to adder 215 byenabling gate 214.

ln response to the fourth pulse on path 256 gate 223 is enabled to applythe function N to the adder 224 where it is incremented to give (N+l)and stored in a portion of register 222. The signal (N+l) is alsoapplied to divider 220 in response to the fourth pulse.

The fourth gate pulse also enables gate 218 so that the complement of xis applied to the adder 217 where it is summed with the output of adder215.

The fifth pulse on line 257 actuates the divider 220 to provide anoutput for the input to adder 230 which is the quotient of the dividendoutput signal from adder 217 and the divisor output signal from adder224. I t

The sixth gate pulse on line 258 enables gate 232 whereby the signal x,is read out of register 236 and added to the output of divider 220thereby producing the outputx At the same time the gate 234 is enabledwhereby the valuex is applied to register 236 with the timing pulse socontrolled that the new signal stored in register 236 does not reachadder 230 in the contemporary clock cycle.

During the final operation the seventh pulse on line 259 enablesselector 208 whereby the work stored in register 241 is placedthrou ghthe selector 208 into the same address in sophisticated arrangements maybe employed. For example it is assumed that register 236 is of suchnature that reading the signals therefrom destroys itscontentsJ-Iowever, register 236 may be of the type in which the contentsare not erased but are monitored. In such case, gates 240 and 262 andregisters 211 and 241 would notbe required. The arrangement permits astep-by-step sequence of operations.

In general, a seismogram such as contained by tape 200 may be a record 4or 5 seconds long with digital samples produced for each 0.00l secondinterval. Similarly, the desired output record z, based on a prioriinformation as to the nature of the environment in which the record uwas obtained will be of like length and sample intervals. It is to.beunderstood that either of the seismograms u or 2 may be treated on thebasis of a different sampling interval such as 0.002, 0.004 seconds andthe like. However, havingapplied the records a and z to the system ofFIG. 6 there will be stored in the memory 203 seismic signal words whichwill cause the system optimally to filter any new seismogram a which hasthe same statistics as the function u. Thus, if a new record a isapplied to the decoder 202, the system will serve to filter the same. Itshould be noted that, in connection with a filtering operation, thefunctions z,, the complement of x,,,,.and the unity function from source225 are employed only in training. Thus, in filtering the switch 270will be open so that the gates 218, 227 and 260 cannot be enabled bytimer 252. Under such conditions the application of an input u willproduce an optimally filtered seismogram on an output tape 275 whichrecords the signal on the output channel 233.

The foregoing description of the system of FIG. 7 may 'employ serialadders. Such adders are well known in the art and may be of the type,for example, manufactured and sold by Texas Instruments Incorporated ofDallas, Texas and identified as SN7483N (a 4-bit adder), SN5482 (a 2-bitadder), and SN5480 (a 1-bit adder). Suitable parallel adders areavailable and are well known, as are multipliers and dividers. 1

Asabove noted classes of operations to which the optimal nonlinearprocessor is applicable include identification or simulation, control,filtering, smoothing, prediction, modeling, classification. While ageneral understanding of the various operations may exist, they will nowbe briefly described with the aid of FIGS. 8-12 to aid in distinguishingthem. It will be found that each case involves a training phase basedupon operations as above described in connection with FIGS. 1,2,5,6and7.

IDENTIFICATION Identification is the determination of a working modelwhich has the same input/output relationship as the system beingidentified. Once obtained, this model can be used to find optimalcontrol laws, the system's response to various trial inputs,or ananalytical description of a process under study. The

.kinetic equations for a chemical process or the equations describingthe dynamics of an airframe are examples. Thus, identification is theessential first step in control and modelmg.

The training phase for identification is shown in FIG. 5. Note that theinputs to system and processor are identical and that the desired outputof the processor is the actual output of the system. Identification hasthe convenient attribute that is available throughout execution. Thus,by multiplexing, training and execution, time-varying systems can betracked. This would be essential in adaptive control. Also,identification has the desirable property that execution can beperformed with an accelerated time scale. Thus, the response ofthernodel to 100 points or samples of a trial u(t) might be determinedbetween just two points of the real-time training sequence.

FIGURES s and 9CONTROL The processor may be used in control in two ways.In one, the physical system or plant is identified in real time.Simultaneously, by means of multiplexed operation, the response of themodel to various trial inputs is established by executing in anaccelerated time scale. By an appropriate iterative procedure, thesequence of trial inputs can be adjusted so that a selected performanceindex is maximized. Then this input can be applied to the plant (in realtime) to effect the optimal control. In this procedure, the processormay provide the service of adaptively tracking the plant and providing amodel which may be operated as fast as the system permits.

Another approach to optimal control is best explained in terms of FIGS.8 and 9. The objective is to produce the controller 300. The latter is asystem which operates on z(t) to give the u(t) which most nearly drivesthe plant 301 to z(t). The optimal nonlinear processor provides this byidentifying the inverse of the plant 301. In general terms, the inverseS of a system S has the property that, when cascaded with the system,the output of the cascaded combination is equal to the input. This isprecisely what is required of the controller 300. An important propertyof S is that it commutes with S, that is, the order of the cascadedcombination is immaterial. This allows the controller to be identifiedas shown in FIG. 9. Note that the processor 302 is required to producean estimate of the input to the plant 301 from its output. If thetraining is successful, it meets the definition of the inverse of theplant, and by the commutivity, property, FIG. 9, may then be installedas the controller 300 as shown in FIG. 8. Systems in which there is nota one-to-one relationship between the input and output sequences willnot have realizable inverses. Even when a true inverse of the plant doesnot exist, the processor will identify the best possible approximation.

FIGURES 7 and 10-FILTERING Filtering is of paramount importance incommunications systems, tracking of airborne vehicles, navigation,secure communications, and many other applications. The objective is toestimate the present value of a signal from an input which is a functionof both signal and noise. The training phase shown in detail in FIG. 7is depicted in block form in FIG. 10. The signal process 303 and noiseprocesses 304 may be interpreted from a purely statistical viewpoint oras the responses of dynamical systems to random excitation. The latteradmits the interpretation of filtering as the estimation of the state ofthe signal process 303 in the presence of measurement noise usinginformation gained from the combination process 305. For example, thesignal process 303 might be a missile trajectory, the noise process 304might be atmospheric disturbances, and the combination process 305 mightbe tracking system dynamics. Together, the signal, noise, andcombination processes constitute what has been called the messageprocess 306. Only when the signal and noise processes are Gaussian andthe combination process linear are linear processors known to beoptimum. Exceptions to these requirements occur frequently and areresponsible for manifestly better performance with the optimum nonlinearprocessor 307.

FIGURE 11-SMOOTHING Smoothing finds wide use in trajectory analysis,instrumentation, and in the estimation of an originating event fromsucceeding events. The estimation of the firing site of a mortar fromradar tracking data of shell trajectory are examples. Smoothing differsfrom filtering in that the objective is to estimate a past value of thesignal from the input rather than the current value. Again, theinterpretation of the signal as the state of a dynamic process isuseful. For example, in trajectory analysis, the state might correspondto the position coordinates of the object under study.

FIG. 6 in which ployed in the mortar firing site detection problemmentioned 1 earlier.

FIGURE IZ-PREDICTION Prediction is important in many areas includingdata'and speech compression. Thatis ifa predictorcan estimate the nexttransmitted bit,that bitneed; not be transmitted--an identical predictorcan be installed/with a receiverto reconstruct the mesage.

To obtain the optimal, nonlineara-predictor, the processor 307 istrained in much'the same manner as for filtering. But in prediction afuture, rather than current, value of the signal is to be estimated fromthe input. Therefore, a pure time advance between the signal andthedesired output/input to the processor is employed. But, as a puretime advance is physically unrealizable, it is necessary to use analternate approach. The same result can be achieved' with a delay 309 onthe processor input as shown in FIG.-. 12. .While the predictionestimator cannot be updated withficurrent informationas was possible inidentification, it can nevertheless'be continually updated as futureevents become'kn'own present-events. Thus, the predictor can copewith.nonstationarysituations' which vary slowly with respect to theprediction lead time. The output-of the predictor is an estimate of-5(1+T) where, in analogy to smoothing, the lead time T can be. eitherfixed or variable. Asan illustration of a variable T. thcichoice T=t -tyields an estimate of .r(t,) which becomes more refined as additionaldata become available, i.e., as the:pr edicted event approaches reality.This might correspond to :the estimate of the impact coordinates of amissile from radarstracking signals beingac-' cumulated.

MODELING 1 'tested. Alternately, certain inputs and outputs may beassumed to act independently. -Thisscan be forced upon the processor andthe assumption testedr.

Once a satisfactory .model..has-.been obtained through identification,it may be examined or otherwise tested to reveal the mathematicaland/orphysical character of the system upon which it is based.

CLASSIFICATION that the objective is not to estimate the signal (or alinear func-' tion of the signal) but to derive a decision based on theesti mated signal. Thus, there will not a 1:1 correspondence between thesignal and desired .outputof the processor, but, the desired output willbe somebinary function of the class to which the signalbelongs. A simplecase is detection shown in w 1, som 0 That is, the processor estimatesthattthe signalis present by producing a unit valued output,or=..thatiitis-not presentby a zero valued output.

nications could be greatly enhanced. Language translation is a closelyrelated problem'The time sequence constituting the input language couldbe classified and converted into a time sequence of code symbols whichdesignates the meaning in the output language. Other classificationswill now be apparent.

Now that the invention as embodied in different forms has been describedin different fields of use it will be appreciated that the method andsystem have general applicability. Operations are not limited to theprocessing of a single valued time varying function such asde scribed inconnection with FIG. 1. They may involve processing a plurality ofsingle valued time varying functions with latitude as to the number offeedback terms and with latitude in the complexity of the system and themethod. The method may be carried out utilizing analog components andsignals as in FIG. 2 or digital components and signals as in FIG. 71:will be appreciated that the method may be performed on a generalpurpose digital computer operating in accordance with the teachingsherein, or on the special purpose digital computer of FIG. 7 sinceanalog operations may in general be carried out effectively in digitalform. In either case, the significant economy'in the storage necessaryduring the training and operating phase provided by the presentinvention obtains and it is therefore to be understood that'theforegoing description is to be taken as exemplary. In most instances,the preferred 'mode of carrying out the invention is in the digital formbut in general will be dependent upon the particular task to which theinvention is to be applied.

From 'the foregoing it will be seen that the invention involves atraining phase for producing a processed output 5 operation which, inresponse to at least two members in each of the groups, will produce anindependent gated training signal; Each such gated training signal iscombined with a contemporary sample of the desired output signal andpreferably with the negative of the previous sample of the processedoutput signal to produce a training summation signal. The independentgated training signal is replaced by the training summation signal'inthe course of each gating operation. The

training summation signal is normalized to reflect the number of timesthat a given independent gated training signal has been selectedorgated. The normalized training summation signal-is then preferablycombined with the previous sample of the processed output signal to formthe contemporary sample of the'processed output signal. The previoussample of the processed output signal is quantized to produce a secondmember of each of the groups which control the gating operation.

The foregoing series of steps is repeated, during training, 'over astatistically significant number of samples of a training input signalor signals.

In a subsequent operating phase, each of a succession of time samples ofthe information signal is quantized to produce a member of eachof a likesuccession of groups of operating signals for controlling the gatingoperation which, in response to at least two'members in each of thelatter groups, will produce an independent normalized gated signal whichwas developed in the training phase. The contemporary normalizedgated'signal is then preferably combined with a previous sample of theprocessed output signal to form a contemporary processed output signal.At the same time the previous processed output signal is quantized toproduce a second member of each of the groups of operating signals tocontrol the gating operation. Thus, during the operating phase thequantities statistically developed during the training phase are appliedin processing any signal which has generally comparable statistics asthe training signal or signals.

The foregoing" description has been directed to operations which may beconsidered to be optimum from the standpoint that the size of the memoryrequired and the error signal experienced in training are minimized.However, it has been found that the system will operate satisfactorilyin somewhat simplified form. For example, the system of FIG. 1 andcounterparts thereof in the remaining figures has been found to operatesatisfactorily even though two of the. three feedback quantities areeliminated. More particularly, the summation unit 30 may be eliminatedso that the output signal x, is the outputof unit 24. That is, thefeedback of the signal x shown in FIG. 1 being applied to the secondinput of summation unit 30 is not used. Further, the feedback paththrough the polarity reversing unit 23 and switch 23a may be eliminated.

The system of FIG. 1 thus modified, gives an error signal which isgreater than when the latter feedback paths are employed but'it isoperable and permits simplification of system which in some cases may befound to be preferable. However, for a given level of performance, alarger memory is required as compared with the system which includes thefeedback to summation unit 30 and to the summation unit 20 as well as toquantizer 14 from delay unit 32. Thus, the system shown in FIG. 1 and asschematically represented in FIG. 2, represents the preferred modeinasmuch as at least three feedback paths are employed, one to quantizer14, the second to the summationunit20, and the third to the summationunit 30. It will be appreciated that the nature of the variouscomponents of the system of FIG. 1 may be varied depending upon theparticular use of the invention. FIG. 2 illustrates an analog storagearray. In digital operations magnetic storage arrays, as conventionallyused in digital systems, may also be employed as well as otherwell-known storage systems. The summations units 20, 26 and 30 may be ofconventional type. In analog operations they have been illustrated inFIG. 2 as comprising resistor networks. In digital operations they wouldcomprise the digital counterpart of the analog, as is well known in theart. Division in divider 24 is well known and may be carried out asdescribed by R. K. Richards in Arithmetic Operations in DigitalComputers, van Nostrand, 1955, pages 136-165. Time delay unit 32 maycomprise a magnetic delay line for either analog or digital operationsor may comprise a storage resistor which will receive and hold a digitalwork for one sample interval as shown at page 153 of the Richards text.The system components employed in the various drawings, therefore, are,in general, well known and understood. The organization of the presentsystem provides for feedback in optimizing the processor of the presentinvention.

Having described the invention in connection with certain specificembodiments thereof, itis to be understood that further modificationsmay now suggest themselves to those 'skilled in the art and it isintended to coversuch modifications as fall within the scope of theappended claims.

What I claim is:

l. The method of establishing an optimal nonlinear signal processorwhich comprises:

a. quantizing successive time samples of an input signal u to producesuccessive groups of operating functions for controlling a gatingoperation which in response to each of said groups will produce anindependent gated output signal; combining said gated output signalproduced in response to the contemporary input signal u; and a functionof the previous output signal ar with a second signal z, representativeof the desired contemporary value of the response of said processor tothe contemporary and past values of u to produce a summation signal:

c. in response to each said gating operation replacing said independentgated output signal by said summation signal;

d. normalizing said summation signal to produce the contemporary valueof the output signal x e.-quant izing said previous output signal x toproduce a second of the groups of said functions to control said gatingoperation.

2. The method of claim 1 wherein said signals, u, x, and z are in analogform.

3. The method of claim 1 wherein said signals u, x and z are in digitalform.

4. The method of claim 1 wherein successive time samples of functionalderivatives of said input signal u and said output at, are employed forfurther controlling said gating operation.

'5. The method of claim 1 wherein said previous output signal x, issubtracted from said first summation signal, and wherein said outputsignal x, is combined with said normalized signal.

6. Optimal nonlinear signal apparatus for processing signals comprising:

a. at least a pair of level selectors the first of which receives saidsignalj b. a pair of storage arrays each addressed in a first coordinateby one of said selectors in response to said signal and each addressedin a second coordinate by the other of said selectors in response to anaddress feedback signal;

c. a first signal loop including a first summation means which firstsummation means is responsive to the condition at the selected addressin a first of said storage means and to a desired output signal forstoring at said selected address the output of said first summationmeans;

d. a second signal loop including a second summation means which secondsummation means is responsive to the condition at said selected addressin a second of said storage means and to a weighting factor for storingat said selected address in said second storage the output of saidsecond summation means;

e. a third feedback loop having in tandem a divider, and a time delaywith the output of said time delay connected to supply said addressfeedback signal to the input of the second of said level selectors;means for applying to said divider the output signals from said first"and second summation means; and

g. circuit means forextracting the processor output signal from thejuncture between said divider and said time delay.

7. The method of producing a processed output signal optimally andnonlinearly from a first signal of a class of signals having similarstatistics which comprises,

i. in a training phase:

a. quantizing successive time samples of at least one training inputsignal other than said first signal selected from said class to producea member of each of successive groups of operating signals partly tocontrol a gating operation which, in response to at least two members ineach of said groups, will produce an independent gated training signal,

b. combining each said gated training signal with a contemporary sampleof the desired output signal to produce a training summation signal,

c. in response to each said gating operation replacing said independentgated training signal by said training summation signal,

d. normalizing said training summation signal to form the contemporarysample of said processed output signal,

e. quantizing said previous sample of said processed output signal toproduce a second member of each of said groups of said operating signalsto control said gating operation, and ii. in a subsequent operationphase: a

f. q'uantizing successive time samples of said first inpu signal toproduce a member of each of successive groups of operating signals forcontrolling said gating operation which, in response to at least twomembers in each of said groups will produce an independent normalizedgated signal developed in said training phase to form the contemporaryprocessed output signal, and

g, quantizing said previous processed output signal to I produce asecond member of each of the groups of said operating signals to controlsaid gating operation.

8 The method of claim 7 wherein, in said training phase, the previoussample of the processed output signal is subtracted from said trainingsummation signal, wherein said previous sample of said processed outputsignal is added to the normalized training summation signal to form thecontemporary sample of said processed output signal, and, in saidoperation phase, each previous sample of the processed output signal isadded to the contemporary normalized gated signal to form thecontemporary processed output signal.

9. Optimal nonlinear signal apparatus for processing signals comprising:

a. at least a pair of level selectors the first of which receives saidsignal;

- b. a pair of storage arrays each being meant for addressing in a firstcoordinate by one of said selectors in response to said signal and in asecond coordinate by the other of said selectors in response to anaddress feedback signal;-

c. a first signal loop including a first summation means which firstsummation means is responsive to the condition at the-selected addressin a first of said storage means and to a desired output signal forstoring at said selected address the output of said first summationmeans;

d. a second signal loop including a second summation means which secondsummation means is responsive to the condition at said selected addressin a second of said storage means and to a weighting factor for storingat said selected address in said second storage the output of saidsecond summation means;

e. a third feedback loop having in tandem a divider and a time delaywith the output of said time delay connected to supply said addressfeedback signal;

f. means for applying to said divider the output signals from said firstand second summation means; and

g. circuit means for extracting the processor output signal from betweensaid divider and said time delay.

10. Apparatus according to claim 9 wherein means are provided forsubtracting the processor output signal from the output of said firstsummation means and for addingvthe processor output signal to the outputof said divider.

11. The method of claim 1 wherein a nonnegative weighting function g isemployed selectively to emphasize said summation signal.

1. The method of establishing an optimal nonlinear signal processorwhich comprises: a. quantizing successive time samples of an inputsignal u to produce successive groups of operating functions forcontrolling a gating operation which in response to each of said groupswill produce an independent gated output signal; b. combining said gatedoutput signal produced in response to the contemporary input signal uiand a function of the previous output signal xi 1 with a second signalzi representative of the desired contemporary value of the response ofsaid processor to the contemporary and past values of u to produce asummation signal; c. in response to each said gating operation replacingsaid independent gated output signal by said summation signal; d.normalizing said summation signal to produce the contemporary value ofthe output signal xi; e. quantizing said previous output signal xi 1 toproduce a second of the groups of said functions to control said gatingoperation.
 2. The method of claim 1 wherein said signals, u, x, and zare in analog form.
 3. The method of claim 1 wherein said signals u, xand z are in digital form.
 4. The method of claim 1 wherein successivetime samples of functional derivatives of said input signal u and saidoutput xi 1 are employed for further controlling said gating operation.5. The method of claim 1 wherein said previous output signal xi 1 issubtracted from said first summation signal, and wherein said outputsignal xi is combined with said normalized signal.
 6. Optimal nonlinearsignal apparatus for processing signals comprising: a. at least a pairof level selectors the first of which receives said signal; b. a pair ofstorage arrays each addressed in a first coordinate by one of saidselectors in response to said signal and each addressed in a secondcoordinate by the other of said selectors in response to an addressfeedback signal; c. a first signal loop including a first summationmeans which first summation means is responsive to the condition at theselected address in a first of saiD storage means and to a desiredoutput signal for storing at said selected address the output of saidfirst summation means; d. a second signal loop including a secondsummation means which second summation means is responsive to thecondition at said selected address in a second of said storage means andto a weighting factor for storing at said selected address in saidsecond storage the output of said second summation means; e. a thirdfeedback loop having in tandem a divider, and a time delay with theoutput of said time delay connected to supply said address feedbacksignal to the input of the second of said level selectors; f. means forapplying to said divider the output signals from said first and secondsummation means; and g. circuit means for extracting the processoroutput signal from the juncture between said divider and said timedelay.
 7. The method of producing a processed output signal optimallyand nonlinearly from a first signal of a class of signals having similarstatistics which comprises, i. in a training phase: a. quantizingsuccessive time samples of at least one training input signal other thansaid first signal selected from said class to produce a member of eachof successive groups of operating signals partly to control a gatingoperation which, in response to at least two members in each of saidgroups, will produce an independent gated training signal, b. combiningeach said gated training signal with a contemporary sample of thedesired output signal to produce a training summation signal, c. inresponse to each said gating operation replacing said independent gatedtraining signal by said training summation signal, d. normalizing saidtraining summation signal to form the contemporary sample of saidprocessed output signal, e. quantizing said previous sample of saidprocessed output signal to produce a second member of each of saidgroups of said operating signals to control said gating operation, andii. in a subsequent operation phase: f. quantizing successive timesamples of said first input signal to produce a member of each ofsuccessive groups of operating signals for controlling said gatingoperation which, in response to at least two members in each of saidgroups will produce an independent normalized gated signal developed insaid training phase to form the contemporary processed output signal,and g. quantizing said previous processed output signal to produce asecond member of each of the groups of said operating signals to controlsaid gating operation.
 8. The method of claim 7 wherein, in saidtraining phase, the previous sample of the processed output signal issubtracted from said training summation signal, wherein said previoussample of said processed output signal is added to the normalizedtraining summation signal to form the contemporary sample of saidprocessed output signal, and, in said operation phase, each previoussample of the processed output signal is added to the contemporarynormalized gated signal to form the contemporary processed outputsignal.
 9. Optimal nonlinear signal apparatus for processing signalscomprising: a. at least a pair of level selectors the first of whichreceives said signal; b. a pair of storage arrays each being meant foraddressing in a first coordinate by one of said selectors in response tosaid signal and in a second coordinate by the other of said selectors inresponse to an address feedback signal; c. a first signal loop includinga first summation means which first summation means is responsive to thecondition at the selected address in a first of said storage means andto a desired output signal for storing at said selected address theoutput of said first summation means; d. a second signal loop includinga second summation means which second summation means is responsive tothe condition at said selected address in a second of said storage meansand to a weighting factor for storing At said selected address in saidsecond storage the output of said second summation means; e. a thirdfeedback loop having in tandem a divider and a time delay with theoutput of said time delay connected to supply said address feedbacksignal; f. means for applying to said divider the output signals fromsaid first and second summation means; and g. circuit means forextracting the processor output signal from between said divider andsaid time delay.
 10. Apparatus according to claim 9 wherein means areprovided for subtracting the processor output signal from the output ofsaid first summation means and for adding the processor output signal tothe output of said divider.
 11. The method of claim 1 wherein anonnegative weighting function g is employed selectively to emphasizesaid summation signal.